In multimedia editing and viewing applications, there are numerous scenarios where a user may want to closely review select portions of multimedia content via scrubbing. Scrubbing is the process of moving within a piece of multimedia content such as a video to locate and present a particular section of the video. The term originally comes from the days of reel-to-reel players, when rocking a reel would give the impression of scrubbing tape across the head. Many video scrub tools today present video in a window and allow the user to drag a cursor across a timeline scrollbar to present different sections of a video to the user. That is, as the cursor is dragged across the scrollbar, the scrub tool updates the display to present that frame of the video represented by the current cursor position. Such scrubbing activities include repeated playback of a small section of content in both forward and reverse directions, displaying individual frames at times controlled by the user, or playback whose rate is finely controlled by the user.
Numerous inter- and intra-frame compression systems compress and decode bitstreams based on groups of pictures (GOP). Each GOP starts with an intracoded (I) frame and includes any number of subsequent forward predictive (P) frames and/or bidirectionally predictive (B) frames—the I-frame is always the first picture in a GOP. To decode any P or B frame in a GOP, decoding always starts with the I-Frame and proceeds to decode each frame upon which the intervening frames depend, until the selected frame is encountered and decoded. Intervening frames are frames between the I-frame and the selected frame. In the context of media decoding, frames that need to be decompressed in order to decompress a particular GOP frame are called “pre-roll frames.”
Thus, the amount of time that it takes to decode a particular frame in a GOP that is not the I-frame is a function of the number of pre-roll frames that need to be decoded to decode the particular frame. In the worst case, the entire GOP must be decoded to decode the particular frame. The length of a GOP is generally based on the encoding settings and the content being encoded. For instance, a video with little or no movement between frames for significant amounts of time may have a GOP that is hundreds and even thousands of frames in length.
Application responsiveness, and by extension responsiveness of the underlying media platform, is crucial in providing a good user experience. Yet, in view of the above, to scrub to a select frame in a GOP, wherein the selected frame is not an I-frame, a scrub tool may have to perform many processing intensive and time consuming decoding operations to reach and decode a selected frame. This substantial limitation does not even take into consideration that after such a selected frame is decoded, the content of the frame may need to be transformed via one or more effects, which will increase the delay even more.
To make matters worse, some media decoders provide forward-only decompression, meaning that reverse playback rates of encoded media comprises of key and delta-frames is very slow. For instance, if a select video portion is to be scrubbed in reverse, the decoding delays already described will exponentially increase. This is because of the reverse iterative I-frame to P/B frame decoding required. (An I-frame is a key frame). In particular, to scrub a portion of video in reverse order, wherein the portion begins with a select P/B frame n, decoding progresses from the I-frame to decode all intervening frames until frame n is reached. Next, the process decodes the I-frame and all intervening frames until frame n−1 is reached. This iterative process continues until the selected portion has been scrubbed in reverse.
Accordingly, systems and methods to improve scrubbing tool performance are desired.